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(54) Position-orientation recognition device 

(57) Upon reception of a woric sensing command 
from a robot controller (1), an image processing unit (2) 
obtains a two-dimensional image of a whole work pallet 
(50) including a work (W). witli a CCD camera (30) [a 
first sensor means]. A line of sight is cak;ulated based 
on the work position on the sensed two-dimensional im- 
age. The robot controller (1 ) determines the destination 
of movement of the robot (40) such that a crossing line 
of slit beams LSI , LB2 of a laser sensor (1 0) [a second 
sensor means) can agree with the calculated line of 
sight. The image processing unit (2) operates the laser 
sensor (1 0) so as to surely irradiate the laser slit beams 
LSI, LB2 to the target work (W). pemnittlng the laser 
sensor (1 0) to sense the three-dimensional position and 
orientation of the taiiget work (W). 
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Description 

[0001] This invention relates to a position-orientation 
recognition device applicable for example to a variety of 
factory automations, and more specifically, to a com- 
pound position-orientation recognition device compris- 
ing a sensor for obtaining two-dimensional Infonnation 
and a sensor for obtaining three-dimensional informa- 
tion. 

[0002] Sensing the position and shape of different tar- 
get works (woricpieces) with various sensors is frequent- 
ly required in assembling and machining operations or 
the like in a production line. A sensor capable of obtain- 
ing a two-dimensional image is generally used for meas- 
urement of the position of the target work on the whole. 
If a CCD camera Is used, for instance, a comparatively 
wide visual field can easily be provided by properly se- 
lecting an optical system (a lens). On the other hand, a 
laser sensor is effectively used for three-dimensional 
measurement of the kx;al position and shape of the tar- 
get work. 

[0003] Laser sensors can be easily attached to a wrist 
portion of a robot and can measure the position and 
shape or the like of details of the target work accurately 
and on a real time basis by irradiating laser beams from 
very close quarters to the target work, when designed 
to take the output from the laser sensor In a robot con- 
troller. For that reason, a laser sensor is frequently 
mounted on a robot system or the like, permitting the 
application to extensive fields Including arc welding, 
sealing and measurement or the like, for instance, 
[0004] The laser sensor is classified into a type to 
scan with a laser beam in the form of a spot beam and 
another type to project a slit beam. In use of either type 
of the laser sensor, it is necessary to Inndlate the spot 
or slit beam onto a target work surface surely. If the tar- 
get work is located at a fixed position with a certain de- 
gree of accu racy and further a position close to the fixed 
position is taught to the robot as an approach position 
in advance, then it Is easy to produce a state where a 
spot beam or slit beam is ready to be irradiated onto the 
target work surface rapidly. 

[0005| However, in many cases according to an ap- 
plication, the position of a target work may remain un- 
known because positioning is not carried out or the ac- 
curacy of positioning of the target woric is not reliable. 
In such a case, It is not possible to start a main meas- 
urement (shape measurement, for instance) with the la- 
ser sensor until the target work is found by search with 
the spot light beam or slit beam of the laser beam as a 
probe, while moving the laser sensor with a transfer 
means such as a robot. 

[0006] In general, laser sensors are not suitable for 
measurement in a wide area, taking time for searching 
for the target work and subsequent approaching to the 
target work. According to the ctrcumstances, it may be 
difficult to cause a laser sensor to search a target work 
surely. 
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[0007] An embodiment of the present inventk>n may 
provide a recognition device which makes it possible to 
use a sensor such as a laser sensor only applicable to 
sense a relatively narrow area, and besides; pemnits the 
s sensor to surely recognize two-dimensional or three-di- 
mensional Infonnation of a target work in a proper posi- 
tion and orientation, even if the position and orientation 
of the target work remain unknown. 
[0008] A recognition device according to the present 
10 invention includes, as prerequisite components, a first 
sensor means for obtaining two-dimensional infomna- 
tlon within a target area and a second sensor means for 
obtaining information within a relatively nan^wer target 
area than the above target area, in the present inven- 
ts tion, the Infonmation obtained by the second sensor 
means is two-dimensional and/or three-dimensional in- 
fo mnation. 

[0009] According to the basic characteristkjs of the 
present invention, the recognition device comprises 

^ means for finding a spatial straight line connecting a tar- 
get work with the first sensor means based on the posi- 
tion and orientation of the first sensor means and the 
two-dimensional infomnatlon including information on a 
target woric, obtained by the first sensor means, means 

^5 for finding such position and orientation of the second 
sensor means, which permits the second sensor means 
to recognize the target work, from the above straight 
line, and nrteans for moving the second sensor means 
to the position and orlentatbn thus found. 

30 [0010] According to the recognition device of the 
present invention, the second sensor means obtains 
two-dimensional information of a target work or three- 
dimensional position and orientation information of the 
target work to recognize the two-dimensional or three- 

35 dimensional infonmation of the target work, after the 
transfer means moves the second sensor means to the 
found position and orientation. 
[0011] When the second sensor means can obtain 
two-dimensional Information, the first sensor means 

40 may be also used for the second sensor means, or vice 
versa. The second sens means may have a light pro- 
jecting means and a photo-detection means and may 
be also provided with a functton for three-dimensional 
measurement of the position and orientation. 

45 [0012] In this case, the photo- detection means may 
be provided as a two-dimensional sensor for obtaining 
the two-dimensional Infonnation, and the second sensor 
means is also allowed to serve as the first sensor 
means. 

50 [0013] When the second sensor means has a light 
projecting means and a photo-detection means and is 
provided with the function for three-dimensional meas- 
urement of the position and orientation, the position and 
orientation of the second sensor means for recognition 

55 of the target work may be detennined such that the di- 
rection of irradiation from the light projecting means 
agrees with the above found straight line (the straight 
line connecting the target woric with the first sensor 
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means) or that the direction of an optical axis of the pho- 
to-detection means agrees with the above straight line. 
[001 4] I n eith er case described above, means for ob- 
taining the spatial straight line or means for finding the 
position and orientation of the second sensor means 
may be provided in a robot controller of an industrial ro- 
bot. Further, the industrial robot may be used as the 
means for moving the second sensor means to the po- 
sition and orientation found as a suitable place for rec- 
ognition. 

[001 5] An important point may be that after finding the 
spatial (actually three-dimensional spatial) "straight 
line" connecting the first sensor means with the target 
wortc in case of searching for the target woric with the 
first sensor means capable of obtaining the two-dimen- 
sional infonnatlon within the relatively wide area, "the 
found straight line" is used for determining "the position 
and orientation suitable for recognition (measurement) 
of the target work" for the second sensor means su itable 
for obtaining the two-dimensional or three-dimensional 
Infomnation within the relatively nanx)w area. 
[001 6] According to the specific procedure, the recog- 
nition device recognizes the position of the target work 
on the two-dimensional image of the target area by firstly 
otstalning the two-dimensional image of the target area 
with the first sensor, and then processing the obtained 
two-dimensional image to find the target work on the 
two-dimensional image. In the present invention, "the 
target area" means an area sufficient to cover a range 
supposed to contain a possibility of the existence of the 
target woric 

[0017] It comes out In this stage that the (actually 
three-dimensional) spatial position of the target wori< is 
on a " line of sight" with which the camera of the first 
sensor seizes the target woric. On the other hand, the 
distance between the camera and the target work on the 
line of sight remains unknown. The "line of sight" may 
be determined based on data obtained by camera cali- 
bration of the first sensor as a straight line connecting 
the center of a camera lens with an imaging position of 
the target woric on the imaging plane as known per se. 
[0018] That is, the relation between the "Imaging po- 
sition of the target work on the imaging plane" or the 
"position of the target work on the image" and the "po- 
sition and direction of the line of sight" can be recog- 
nized based on the data obtained by camera calibration. 
[0019] Generally speaking, if a position (u, v) on an 
Image (two-dimensional) space is found, then the line 
of sight (a straight line) passing through the position is 
found. An equation given by the line of sight (the straight 
line) is also called an "equation of a line of sight". 
[0020] Accordingly, the line of sight can be used as 
the "straight line connecting the first sensor means with 
the target woric". Strtetly speaking, it should be noted 
that the "position of the target woric" means "the position 
of a certain point (which will be referred to as a repre- 
sentative point) on the target woric, and as a result, 
the "straight line connecting the first sensor means with 



the target woric" is equivalent to a "straight line connect- 
ing the first sensor means with the representative point 
on the target work". 

[0021 ] In this connection, the "straight line connecting 
5 the first sensor means with the target work" obtained as 
described above is effectively utilized for determining 
the position and orientation of the second sensor which 
is suitable for more detailed recognition (of more accu- 
rate position, orientation, shape and dimension or the 
10 like of the target work). 

[0022] Because, as the target work surety exists on 
the "spatial straight line" thus found, the target work can 
be sensed surely if the second sensor Is moved to a po- 
sition on the spatial straight line or a position close to 
IS the spatial straight line and the visual field of the second 
sensor Is oriented in the direction matching the above 
straight line; 

[0023] For instance, if the second sensor has a light 
projecting means and a photo-detection means and is 
20 provided with the function for three-dimensional meas- 
urement of the position and orientation, it is enough to 
determine the orientation of the se(X)nd sensor based 
on the direction of irradiation or the direction of the op- 
tical axis of the photo-detection means. 

25 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0024] The foregoing and other features of the inven- 
tion will become apparent from the following description 
30 of preferred embodiments of the Inventnn with refer- 
ence to the accompanying drawings, In whteh: 

Fig. 1 illustrates an outline of the overall arrange- 
ment of a three-dimensional position-orientation 
35 recognition device according to a first embodiment 
of the present Invention; 

Fig. 2 illustrates an outline of the configuration and 
operation of a laser sensor, 
Rg. 3 Is a view showing the positional relation 
40 among two planes carrying two laser slit beams 
LB1 , LB2, a crossing tine LC of these two laser slit 
beams, a laser beam irradiation sec^tion and a pho- 
to-detection section or the like; 
Rg. 4 Is a flow chart for explaining a process of the 
45 operation of the position-orientation recognition de- 
vice of Fig. 1 ; 

Fig. 5 is a view showing a two-dimensional image 
obtained by a first sensor means in the position -ori- 
entation recognition devtee of Fig. 1 ; 
90 Fig. 6 is a view showing the relation between a line 
of sight and a target work; and 
Rg. 7 illustrates an outiine of the overall arrange- 
ment of a three-dimenstonal position-orientation 
recognition device according to a second embodl- 
ment of the present invention. 

P)025] A descriptbn will be firstly given of an outiine 
of a three-dimensional position-orientation recognition 
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device according to a first embodiment of the present 
Invention with reference to Rg. 1 . This three-dimension- 
al position-orientation recognition device is applied to 
handling (pick-up and carriage) of works by an Industrial 
robot (which will be hereinafter simply refen-ed to as a 
robot), for instance. In the present embodiment, it Is as- 
sumed that a plurality of works are placed at different 
positions and orientations. This position-orientation rec- 
ognition device fulfills Its characteristics in starting han- 
dling the works placed in the above state. 
[0026] As shown In Rg. 1 , the position-orientation rec- 
ognition device comprises a robot controller 1 , an Image 
processing unit 2. a laser sensor body section (which 
will be hereinafter simply refen^ed to as a laser sensor) 
1 0, a laser sensor control section 20. a CCD camera 30 
and a robot 40. 

[0027] The laser sensor 1 0 is mounted to a wrist por- 
tion (in the vicinity of a tool center point (TCP)) of the 
robot 40. The CCD camera 30 is Installed above a work 
pallet 50, for Instance, without being mounted to the ro- 
bot 40. The CCD camera 30 and the image processing 
unit 2 constitutes a wide area sensor (a first sensor 
means) . For that reason, the installation position and 
orientation of the CCD camera 30 are detenmined such 
that a visual field 31 of the CCD camera may cover the 
entirety of the work pallet 50 provMed as a supply area 
of a woric W. 

[0028] It is assumed that calibration of the CCD cam- 
era 30 for obtaining infomriation on the relation between 
a position (u. v) on an image (two-dimensional) space 
and a line of sight (a straight line) in advance has been 
already finished. Data representing the relation between 
a position (u, v) on an image and a position on a three- 
dimensional coordinate system (an appropriately set 
woric coordinate system) is stored In a memory in the 
Image processing unit 2 in advance. 
[0029] The laser sensor 1 0 used as a nan-ow area 
sensor (a second sensor means) may be of either well- 
known type of spot beam Inradiation or slit beam irradi- 
ation. However, a description will now be given of the 
position-orientation recognition device in case of using 
the sensor of the latter type with reference to Rg. 2. 
[0030] A laser beam irradiation section 1 3 of the laser 
sensor 1 0 has first and second laser oscillators 11,12. 
while a photo-detection section (a photo-receiving sec- 
tion) 1 4 has a photo detector 14a and an optical system 
14b for Image fonmation. The first and second laser os- 
cillators 11,12 output first and second silt beams LB1 . 
LB2 respectively through a cylindrical lens placed in the 
vicinity of each in-adiatlon opening. 
[0031] The laser sensor 1 0 is connected to the image 
processing unit 2 through the laser sensor control sec- 
tion 20. The CCD camera 30 Is also connected to the 
image processing unit 2. As described above, ttie image 
processing unit 2 is used by the laser sensor (the body 
section 10 and the control section 20) and also by the 
CCD camera 30. 

[0032] Upon reception of a laser sensor operation 



command from the Image processing unit 2 through a 
line 24, first and second laser drive sections 21 . 22 start 
driving the first and second laser oscillators 11, 12 to 
generate first and second laser slit beams LB1 . LB2. 

5 [0033] Laser beams subjected to diffuse reflection 
from a reflection point S1 (a point of incidence of a de- 
sired single beam L1 in the first slit beam LB1) and a 
reflection point S2 (a point of Incidence of a desired sin- 
gle beam L2 in the second slit beam LB2) on the surface 

10 of the target work form an image on the photo detector 
14a by means of the optical system 14b according to 
the positions of the reflection points SI , S2. CCD or the 
like is applteable to the photo detector, in the present 
embodiment, a two-dimensional CCD array Is used as 

15 the photo detector 1 4a. 

[0034] Rg. 3 shows the positional relation among two 
planes carrying the first and second laser silt beams 
LB1 , LB2, a crossing line LC of these laser beams, the 
laser beam irradiation section 13 and the photo-detec- 

^0 tion section 1 4 or the like. It is assumed that calibration 
of ttie laser sensor 10 has been already finished, and 
its result (calibration data) is stored in the memory in the 
Image processing unit 2 in advance. The calibration data 
on the laser sensor 1 0 contains data on the position and 

25 orientation (fixed values) of each of the first and second 
planes carrying the first and second laser slit beams 
LB1 . LB2 with respect to the body of the laser sensor 
1 0. The crossing line LC can also be calculated from the 
data on these planes at any time. 

30 [0035] Accordingly, if the relation between the coordi- 
nate system set on the body of the laser sensor 1 0 and 
the coordinate system set on the robot (the robot coor- 
dinate system, that is, the coordinate system fixed to the 
arm end of the robot 40, for instance) is obtained in ad- 

35 vance, the position and direction on the robot coordinate 
system relating to each of the first and second planes 
canying the first and second laser slit beams LB1 , LB2 
and the crossing line LC may be established. As well 
known per se, tiie above relation Is generally expressed 

40 in the forfn of a 4 X 4 homogeneous ti'ansfomrtation ma- 
trix, which is found from tiie calibration of the laser sen- 
sor. In this stage, the calibration of the laser sensor has 
been already finished, and the calibration data Is stored 
in the memory in the image processing unit 2 In ad- 

45 vance. 

[0036] A description will now be given of a process of 
tiie operation of the position -orientation recognition de- 
vfee according to the present embodiment with refer- 
ence to Figs. 4 to 6. 

50 [0037] The image processing unit 2 is waiting for a 
work sensing command to be sent from the robot con- 
troller 1 (Step SI). If the Image processing unit 2 re- 
ceives this woric sensing command, a processing se- 
quence for sensing the work is started to output a pho- 

55 tographing command to tiie CCD camera 30 (Step S2). 
Subsequently, a two-dimensional image of the whole 
work pallet 50. Including an image of the woric W. Is ob- 
tained for storage Into the frame memory in the image 
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processing unit 2 (Step S3). 

[0038] Fig. 5 shows an example of the image thus ob- 
tained. In Fig. 5, reference numeral 70 denotes a repre- 
sentative point (a center of gravity of a Vi^ork contour im- 
age, for instance) of the sensed work W, and the two- 5 
dimensional Image position (u70, v70) of the represent- 
ative point 70 is found with an Image processor in the 
Image processing unit 2. Further, a line of sight 32 (see 
Fig. 6) corresponding to the two-dimensional image po- 
sition (u70, v70) of the representative point is calculated 
from the data obtained by calibration. 
[0039] Since the point 70 representative of the work 
W exists on the line of sight, the position and orientation 
(an equation on the work coordinate system) of this line 
of sight correspond to "(Information on ) the position and 
orientation of the work W" which has been recognized 
in this stage. I n this process, data obtained by transfonm- 
ing the infomiatlon of the position and orientation of the 
work W into data on the robot coordinate system is re- 
quired (Step S4). Incidentally, coordinate system setting 
data (the 4x4 homogeneous transformation matrix 
representing the relation between the robot coordinate 
system and the woric coordinate system) stored in the 
memory of the robot controller 1 is used for the above 
transfomnation. 

[0040] It. Is to be noted in this stage that the position 
of the work W represented by the point 70 remains un- 
certainty. That is, as shown In Rg. 6, although the three- 
dimensional spatial position of the work W (the repre- 
sentative point 70) Is limited to be on the line of sight 32 
according to the principle of perspective transformation, 
the distance between the wori< W emd the camera 30 
remains unknown. Thus, it is not possible to determine 
whether the work W is at a position 75 on the line of sight 
or a position 76 more remote from the camera 30 than 
the position 75, for instance. 

[0041 1 infomnation (robot coordinate system informa- 
tion) of the line of sight 32 is sent to the robot controiier 
1 as infomnation representing "a straight line connecting 
the first sensor means with the target work" (Step S5). 
[0042] Subsequently, the position and orientation of 
the laser sensor 1 0 (the second sensor means) mount- 
ed to the robot 40, which Is suitable to recognize (meas- 
ure) the target work W, are calculated (Step S6). The 
above calculation of the position and orientation may be 
perfonned in the robot controller 1 . One of the require- 
ments used for this calculation is to "cause the line of 
sight 32 to agree with the crossing line LC (see Fig. 3)**. 
An additional condition as to how distant the laser sen- 
sor 1 0 should be placed may by fulfilled by Imposing an 
appropriate condition in advance. 
[0043] The additional condition will be fulfilled by set- 
ting the woric coordinate system so that the X-Y plane 
thereof is placed on the pallet 50 and then setting "the 
Z-coordinate of the position of the robot 40 on the work 
coordinate system to a fixed value ZO". The robot con- 
troller 1 moves the rok>ot 40 to the position and orienta- 
tion calculated under these conditions. Incidentally, in- 



formation specifying Ihe direction" of the crossing line 
LC of the. laser sensor 10 is required. However, this re- 
quirement is fulfilled by imposing a condition on one 
component (on the work coordinate system) of the ori- 
entation of the robot 40 (or by designating a robot ori- 
entation 80 that the laser sensor looks down). 
[0044] On completion of movement of the robot, the 
laser sensor 1 0 Is placed In a ready state for Irradiation 
of the laser slit beams LB1 , LB2 where the crossing line 
LC of these beams will agree with the line of sight 32 
and furtiier the crossing line LC will meet the target wori< 
at the point 70. 

[0045] After the completion of movement of the robot, 
the robot controller 1 infomis the image processing unit 
2 of the completion of movement. Then, the image 
processing unit 2 outputs a photographing command 
(an operation command) to the laser sensor 10 (Step 
87). Then, the laser sensor 10 starts the operation to 
irradiate the laser sirt beams LB1 , LB2, and the crossing 
line LC of these beams meets the target woric at the point 
70 (Step SB). 

[0046] In the present embodiment, as the laser sensor 
1 0 is a three-dimensional sensor, the three-dimensional 
position of the target work represented by the point 70 
can be measured. The orientation of the target work W 
can also be detected from the reflected Image (on the 
photo detector 14a) corresponding to the laser slit 
beams LB1 , LB2 (Step S9). 

[0047] Thus, even If the work W is placed at an in- 
clined posture on another woric or the height of each 
work W Is not unlfomi, the laser sensor 10 can detect 
the height and inclination of the work and so on. Accord- 
ingly, the position and orientation of the grasping portion 
in the robot 40 can be controlled based on the height 
and orientation of the woric W. 
[0048] Incidentally, if the work W was once detected 
on the crossing line LC of the laser sensor 1 0, more de- 
tailed infomnation can be obtained easily by changing 
the position or the orientation of the robot 40 slightly and 
then carrying out measurement under a c»>ndition where 
the crossing line LC is incident on a point other than the 
point 70. 

[0049] A description will now be given of the outline 
of a three-dimensional position-orientation recognition 
device according to a second embodiment of the 
present Invention with reference of Fig. 7. The device of 
the second embodiment is suited for a case where the 
size of a pallet 50 (the range to be covered with the wide 
area sensor) is not quite targe. 
[0050] A difference between the second embodiment 
of Fig. 7 and the first embodiment of Fig. 1 is that the 
camera 30 is mounted on the robot 40 as a photo-de- 
tection section 1 40 of a laser sensor 11 0, in the case of 
the second embodiment, to meet a requirement that the 
photo-detection section should also serve as the first 
sensor means. A laser beam irradiation section 130 of 
Rg. 7 is similar to the laser beam irradiation section 13 
of the laser sensor 1 0 of Fig. 1 . As for the other compo- 
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nents, the device of Fig 7 and that of Fig. 1 have the 
common structures. 

[0051 ] However, system operating sofhware stored in 
the Image processing unit 2 or the robot controller 1 is 
altered such that the CCD camera 30 of the laser sensor 
110 can be switched between a first mode (similar to 
that of the first embodiment) used for obtaining the two- 
dimensional Image at the time of photographing and a 
second mode for sensing the reflected light of the laser 
slit beam from the laser sensor 110. Further, it is as- 
sumed that calibration of the photo-detection section 
(note that this photo-detection section is provided with 
an Imaging lens) 140 has already been finished similarly 
to the calibration of the CCD camera 30, and that data 
obtained l^y calibration has already been stored in a 
memory of the image processing unit 2. 
[0052] The outline of a process of the operation of the 
position recognition device according to the second em- 
bodiment is described below. IHowever, it should be not- 
ed that description of portions which are the same as 
the operations of the position-orientation recognition de- 
vice according to the first embodiment will not be repeat- 
ed here as far as possible. 

[0053] When the image processing unit 2 receives a 
work sensing command from the robot controller 1 , an 
operation command in a nonmal photographing mode Is 
outputted to the laser sensor control section 20. Then, 
a two-dimensional image of the whole work pallet 50, 
including an image of the work is obtained for storage 
into a frame memory in the innage processing unit 2. 
[0054] Fig. 5 shows an example of the image thus ob- 
tained. Similarly to the above first embodiment, the line 
of sight 32 is calculated based on the position (uZO, v70) 
of the representative point 70 on the two-dimensional 
image by using data obtained by calibration. 
[0055] Infonnnation (on the robot coordinate system) 
of the line of sight 32 (that is, an optical axis of the photo- 
detection section 140 of the laser sensor 1 10) is sent to 
the robot controller 1 as Infonnation representing "a 
straight line connecting the first sensor means with the 
target work". 

[0056] Subsequently, the position and orientation of 
the work W with the laser sensor 1 30 (the second sensor 
means) mounted to the robot 40, which is suitable to 
recognize (measure), is calculated. This calculation 
may be perfomned in the robot controller 1 . The condi- 
tions to be applied to this calculation may be similar to 
the conditions to be applied to the calculatton In the case 
of the first embodiment. That is, those conditions include 
a requirement that "the line of sight 32 should agree with 
the crossing line LC (see Fig. 3)" and an additional con- 
dition for determining how distant the laser sensor 110 
should be placed. 

[0057] Upon completion of movement of the robot, the 
robot controller 1 informs the image processing unit 2 of 
the completion of movement, and the Image processing 
unit 2 outputs a photographing command in a laser silt 
beam photographing mode to the laser sensor 110. 



Then, the laser sensor 110 starts operation to irradiate 
the laser slit beams LB1 , LB2, and the crossing line LC 
meets the target work 70 at the point 70. The reflected 
light of the laser slit beams LB1 , L62 fonns an image on 

5 the photo detector of the CCD camera 30. The image 
processing unit 2 obtains the three-dimensional position 
and orientation of the target work W after analyzing the 
result of sensing according to the sofhvare processing 
for performing the nomrial functions of the laser sensor 

10 110. 

[0058] Although the three-dimensional sensor is used 
as the second sensor means In the above two embodi- 
ments, it is to be understood that a two-dimensional sen- 
sor can also be used depending on the use. Even in a 

IS case where such a two-dimensional sensor is em- 
ployed, the position and orientation of the-second senor 
means suitable for recognition of the target work are de- 
termined based on the "straight line connecting the tar- 
get work with the first sensor" in the same way as in the 

20 case where a three-dimensional sensor is employed. 
[0059] Although the robot is used as the transfer 
means of the sensor In the above embodiments, It Is to 
be understood that other transfer means which can con- 
trol the position and orientation of the sensor (for exam- 

2s pie, a XYZ table of which orientation is controllable) can 
. also be used. 
[0060] According to the present invention, even in a 
case where the position, height and orientation of the 
target work are not specified, the position and orienta- 

30 tion of the second sensor means for recognition can be 
property determined based on "the straight line connect- 
ing the target woric with the first sensor" determined by 
using the two-dimensional image obtained by the first 
sensor means. Thus, the position-orientation recogni- 

3s tion devce of the present invention makes much for au- 
tomation of various operations. 



Claims 

40 

1. A position-orientation recognition device including 
a first sensor means for obtaining two-dimensional 
information within atanget area and a second sen- 
sor means for obtaining two-dimenstonal or three- 
dimensional Infomiation within a relatively nan-ower 
target area than said target area, comprising 

means for finding a spatial straight line con- 
necting a target work with said first sensor 

so means based on two-dimensional infonDation 

including infonnation on a target work, obtained 
by said first sensor means, and the position and 
orientation of said first sensor means; 
means for finding such position and orientation 

ss of said second sensor means, whk;h permits 

said second sensor means to recognize said 
target woric, from said straight line; and 
means for moving said second sensor means 



6 



11 



EP1 190 818 A2 



12 



to the position and orientation thus found; 

wherein said second sensor means ob- 
tains two dtnnensional Infonnation of the target 
work or three-dimensional position and orien- 
tation Infomnation of the target work to recog- 
nize the two-dimensional or three-dimensional 
information of the target work, after said trans- 
fer means moves the second sensor means to 
said found position and orientatk>n. 

2. The position-orientation recognition devtee accord- 
ing to claim 1 , wherein said second sensor means 
is designed to obtain two-dimensional infomnation 
and is also used as the first sensor means. 

3. The position-orientation recognition device accord- 
ing to claim 1 or 2, wherein said second sensor 
means has a beam projecting mans and a photo- 
detection means and is provided with a function for 
three-dimensional measurement of the position and 
orientation. 

4. The posttion-orlentation recognition device accord- 
ing to claim 3, wherein said photo-detection means 
of said second sensor means is a two-dimensional 
sensor for obtaining two-dimenslonai infomnation. 

5. The position -orientation recognition device accord- 
ing to claim 4, wherein said second sensor means 
serves as the first sensor means. 

6. The position-orientation recognition device device 
according to claim 3, 4 or 5, wherein the position 
and orientatbn of said second sensor means for 
recognizing said target work are detemilned such 
that the direction of irradiation from said beam pro- 
jecting means agrees with said spatial straight line. 

7. The positton-orlentatton recognltton device accord- 
ing to any preceding claim, wherein the posltton and 
orientation of said second sensor means for recog- 
nizing said target work are detemriined such that the 
direction of an optical axis of said photo-detection 
means agrees with said spatial straight line. 

8. The position-orientation recognition device accord- 
ing to any prec claim, wherein the means for finding 
said spatial straight line orthe means for finding the 
position and orientation of said second sensor 
means is provided in a robot controller of an indus- 
trial robot. 
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defining a straight line connecting a first sensor 
for obtaining two-dimensional infomnation with 
a target woric for recognition in a three-dimen- 
sional space, by using said first sensor; 
moving a second sensor for obtaining three-di- 
mensional inforniation within said space up to 
the position and orientation enough to recog- 
nize the target wori( according to infomnation of 
said straight line: and 

obtaining three-dimensional information of said 
target work with the second sensor which has 
reached the position and orientation enough to 
recognize said target work. 

11. A position-orientation recognition device operable 
to perfomi the steps of: 

defining a straight line connecting a first sensor 
for obtaining two-dimensional Infomnation with 
a target work for recognition in a three-dimen- 
sional space, by using said first sensor; 
moving a second sensor for obtaining three-di- 
mensional information within said space up to 
the position and orientation enough to recog- 
nize the target woric according to Infomnation of 
said straight line; and 

obtaining three-dimensional Infomnation of said 
target work with the second sensor which has 
reached the position and orientation enough to 
recognize said target work. 
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9. A position-orientation recognition device according 
to any preceding claim, wherein said transfer 
means is an Individual robot. 
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10. A method forth ree-dlmensional position-orientation 
recognition, comprising the steps of: 
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